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I. INTRODUCTION 



Computer graphics images have large data storage requirements. For example, 
the video memory bitmap' for a computer monitor with a resolution of 1024 by 1024 
pixels requires one megabyte of memory to store an eight-bit grey-scale image. A 32- 
bit color image consumes four times the storage resource. The time taken to send such 
an image through a communication channel is significant. Therefore, methods which 
can reduce the size of a graphics file are of practical interest. 

A. BACKGROUND 

In the recent past, it was expected that graphics output and programs created on 
one computer would be used exclusively within the realm of that environment. Now, 
with the increasing popularity of computer networks, the capability exists for sharing 
resources among different computer systems. Indeed, Tanenbaum suggests that one 
goal of networking is "to make all programs, data, and other resources available to 
anyone on the network without regard to the physical location of the resource and the 
user." [TanSl: p.3] No longer is the computer user limited to software, hardware, and 
peripherals designed only for his particular machine. 

The tremendous growth in the field of database management has meant an 
increase in large-scale information transfer by remote computing and the development 
of massive information storage and retrieval systems. Any method which reduces the 
size of the data for such systems implies a savings in the cost of data storage ^d 
transmission time. Thus, data compression techniques have gained popularity as a 
realistic means to accomplish these savings. [Hel87: p.l] 

Likewise, graphics applications have become more sophisticated and gained 
popularity in networking environments. These applications consume substantial 
computer resources such as memory and processor time. It is often desirable to use 



' Refer to Appendix A, Glossary, for a brief definition of unfamiliar tenns. 



1 



mainframe resources to develop and perhaps store graphics, but to be able to view, 
save, and manipulate the graphic image on a personal microcomputer or workstation. 

Because graphics files are often very large, the process of sending them from a 
host computer to a microcomputer via low bandwidth channels (such as phone lines) 
can be slow. If a user is interested in rapidly viewing many files (a graphics database, 
for instance) the time required to transmit the files may seem unreasonable. Current 
research in methods of compression are proving beneficial in significantly reducing the 
size, and therefore, the time to transmit an image. 

B. OBJECTIVE 

The goal of this thesis is to examine and evaluate several methods of graphic 
data compression which are used in the field of computer science. In addition, we wUl 
look at these methods in relation to transmitting graphic images from the Naval 
Postgraduate School’s IBM 3033 mainfi'ame computer to microcomputers in order to 
determine a reasonable method of reducing the image transmission time. 

C. SCOPE OF THE THESIS 

Graphics may be stored as vector images or as bitmaps. This thesis will only 
address graphic images stored in bitmapped format. Research in various methods of 
reducing the size of the transmitted image file fall into these categories: (a) 

compression methods of passing all the information, but taking fewer bits to do so 
("lossless" compression) or (b) methods of reducing the amount of information passed 
through the network and reconstructing an incomplete image on the receiving computer 
("lossy" compression). This thesis wUl investigate only the first category with emphasis 
primarily on techniques which can be applied specifically to the compression of binary 
(black / white) images. 

D. OVERVIEW 

Chapter II introduces several compression techniques in use in the field of 
computing. In subsequent chapters, these are discussed in depth, showing examples 
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of current use of these compression methods. Emphasis is on techniques used to 
compress graphical data. 

Chapter HI discusses the run-length encoding technique. Chapter IV explores 
statistical coding with particular emphasis on Huffman codes. Chapter V concerns 
other techniques, especially that of relative encoding. 

Chapter VI is an the evaluation of the techniques discussed in the previous 
ch£q)ters. The main focus is on how these techniques can be applied to binary images. 

Chapter Vn describes the specific application of one technique of transmitting 
binary images in a network environment. The question is, "Can the compression 
techniques studied reduce the time to transmit digital images from the IBM 3033 
mainframe to the IBM microcomputer by significantly compressing the file to be sent?" 

Chapter VUI offers a summary of research findings and conclusions drawn from 
applying these techniques to the issue of binary image transmission. 

A set of appendices is included to aid the reader in understanding details of the 
thesis. Appendix A is a glossary of terms which may not be familiar to the reader. 
Appendix B displays, in reduced format, the graphs which are included in the sample 
set used in the compression implementation in Chapter VII. Appendix C contains a 
listing of the programs written to perform the Huffman coding and to analyze the 
compression results. And Appendix D shows an example of an RLE (run-length 
encoded) file and the Huffman coding of the file. 
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II. DATA COMPRESSION METHODS 



Davisson and Gray define data compression as "the science and/or art of 
massaging data from a given information source in such a way as to obtain a 
simplified or compressed version of the source data with at most some tolerable loss 
of fidelity." Areas where data compression has been used include systems for 
communications, speech and image processing, pattern recognition, information retrieval, 
storage, and cryptography. But the theory and practice of data compression began as 
early as 1898, when W. F. Sheppard studied the "rounding off of real numbers to a 
fixed number of decimal places. [Dav76: p.l] 

Because data compression, the substitution of data by a more compact 
representation, implies savings of resources (transmission time, storage media, computer 
memory, and money), it will always be a topic worthy of study. In the middle of this 
century. Shannon [Sha48], Fano [Fan49], and Huffman [Huf52] were researching 
improved methods of data compression. 

In 1977, the National Bureau of Standards published a report on data compression 
to assist Federal Agencies in developing economical data element standards [Aro77: 
p.l]. In 1988, a paper published in ACM Computing Surveys assessed a variety of 
data compression methods spanning almost 40 years of research [Lel88]. Many of the 
techniques covered in both papers were based on the earlier work of Shannon, Fano, 
and Huffman, a tribute to the continued importance of these methods of data 
compression. 

A. EXPLANATION OF BITMAPS 

Basically there are two ways to represent a graphic in computer memory: as a 
vectorized image or as a bitmapped image. The first method .stores the graphic 
information as sets of coordinates of the lines, or vectors, which define the image. The 
second method stores a bitmap of the image in memory. TTiis thesis is limited to the 
domain of bitmapped graphic images. 
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A bitmap is a virtual representation of a specific screen image of the target 
monitor. For instance, an Enhanced Graphics Adapter (EGA) monitor which has a 
resolution of 640 pixels in the horizontal direction and 350 pixels vertically, contains 
a total of 224,000 pixels. If the monitor is monochrome, then each pixel has only two 
states, on or off, and may be represented by one bit in memory. TTie bitmap (bitplane, 
or monitor mapping) occupies 28 kilobytes of computer memory. The mapping of a 
color monitor is different. 

Each pixel of a color monitor is composed of three colors: red, green, and blue. 
The intensity of each color varies to define different hues, while the combination of 
the intensities of all three colors designates the shades of pixel color. For instance, an 
IRIS-4D GT monitor has a resolution of 1280 by 1024 pixels. Each color may be set 
to 256 different intensities, requiring eight bits per color or 24 bits per pixel. Thus 
there are 16 million different colors available on this system. 

How color graphics is implemented varies greatly from system to system. Even 
grey-scale graphics may take from eight to 24 bits to represent one pixel. Values 
range from black to white; black is represented by three values of the lowest intensity 
(0,0,0), white by three values of the highest intensity (256,256,256), and grey shades 
in between by equal values of mid-range intensities (100,100,100). [SGI: p.4-3] 

In the computer graphics monitor industry, the number of horizontal and vertical 
addressable pixels on the CRT is called resolution. Resolution depends on the pitch 
and size of the phosphor dots on the CRT screen, and the brighmess and purity of 
color that can be displayed. The size, spacing and number of dots is a function of the 
tube, but brightness and color quality depend on the monitor’s electronics. 

The following quotation shows the range of differences in monitors on the market 
today and indicates the varied hardware and software that must exist to support such 
diverse configurations. 

TTte 19-in. CRT is the de facto standard display size for engineering 
workstations. The large viewing area offers reduced eye strain. A monitor with 
a tube this size is considered to be ultra-high resolution if it has more than 1,280 
horizontal addressable pixels. An example is 1,600 horizontal by 1,280 vertical. 
But if it has 1,000 horizontal pixels or above, it is considered to be a high 
resolution. A monitor with a 1,024 by 800 resolution is an example of this level. 
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The most popular monitors for PCs used in CAD/CAM and graphics 
application have screens measuring 13 or 14 inches diagonally. A monitor in this 
size range with 500 to 1,000 horizontal pixels is considered to be a high 
resolution. Sony, for example, offers a 900 by 560 monitor while NEC offers 
an 800 by 560 product. 

...Among the state-of-the-art monitors now on the market are the Sun-4 
workstation with 1,152 by 900 resolution and IBM’s PS/2 system monitor with 
1,024 by 768 resolution.... 

...Sony, received a custom contract from the FAA for a 37-in. graphics 
monitors to be used in upgrading the U.S. air traffic control system. ...The 
custom-made monitors with 2,000 by 2,000 resolution reportedly sold for $50,000 
each. [Wil88: p.lO] 

Figure 2.1 shows typical resolutions for some of the popular graphics adapters 
on the market. 



Low 


Resolution (LR) : 


128 X 128 to 510 X 510 


Text 


only 


MDA 


320 


X 240 


CGA, MCGA 


Medium Resolution (MR) 


: 512 X 512 to 800 x 600 


640 


X 350 


EGA 


512 


X 512 




852 


X 350 


Super EGA 


720 


X 348 


Hercules 


640 


X 480 


VGA and PGA 


800 


X 600 


Super VGA 


High 


Resolution (HR) ; 


801 X 601 to 1200 X 1023 


1024 


X 768 


8514/A, Extended VGA 


1280 


X 800 


Wyse and other DTP 


Very 


High Resolution (VHR) : 1201 x 1024 to 2048 x 2048 


1280 


X 1024 


IBM' s next controller 


1600 


X 1024 




1680 


X 1280 




1200 


X 1800 


MCA (IBM's future controller) 


Ultra High Resolution 


(UHR) : 2049 x 2049 and above 


3072 


X 2048 


UHR DTP systems 


4096 


X 4096 


Vector displays 



Figure 2.1. Resolution Segments [Ped89: p.8]. 
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B. LOSSLESS AND LOSSY COMPRESSION 



TTiis thesis concentrates on compression techniques referred to as "lossless." 
Using these methods of compression, a file is compressed (encoded), transmitted, and 
decompressed (decoded) to produce a file identical to the original. The methods which 
will be discussed include run-length encoding, statistical encoding, and relative 
encoding. 

Contrast this to methods of "lossy" compression where a graphics file is encoded 
such that an incomplete image is re-created in the decoding phase. Examples include 
fractal image compression, color compression, and spatial compression. 

In fractal image compression, the shapes of natural objects in the original graphic 
image, such as trees, clouds, and fire, are numerically encoded using fractal geometry. 
These are then re-created as fractal images on the target machine. The technique is 
"lossy" because, although the decoded images closely resemble the original shapes, they 
are representations. [Win88: p.24] [Pet87] [Bar88] 

Another method of compressing color images is to reduce the number of bits per 
pixel normally available to a system. This technique limits the maximum number of 
colors from, say, 256 (16-bit pixels) to 16 colors (four-bit pixels), but is effective 
where color does not play an integral part in the identification of the image. A 
satellite image is a good candidate for color compression, whereas a graphic design of 
molecules which are distinguished by their color may not be. [Mur88b] 

In some instances color definition may be important, but a fine resolution may 
not be. Spatial compression takes advantage of this property. Consider a bitmap 
which represents a screen resolution of 512 by 512 pixels, or 256 kilobytes for an 
eight-bit grey-scale image. Each four by four block of pixels is replaced with one 
pixel containing a weighted average of the intensity values of the 16 pixels in the 
original block. Using this technique the bitmap can be reduced to 128 by 128 pixels, 
or 16 kilobytes, although the decoded image may appear fairly ragged. If the .spatially- 
compressed image is recognizable, then the achieved compression ratio is 16:1. This 
type of lossy compression is useful in browsing a large database of graphic images. 
When the desired image is located, it may be transmitted by a lossless method and 
completely reconstructed on the target monitor. [Mur88a] 
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III. RUN-LENGTH ENCODING 



Run-length encoding compresses data by taking advantage of a run, or series of 

the same value occurring consecutively. A run may be any of the following: a 

rep>eating single-bit value in a black and white bitmapped image file, a repeating 

character in a text file, or a repeating pixel value in a color graphics file. A run may 

even be a larger pattern which repeats itself a number of times. For instance, consider 
that a repeating number is actually a repeated pattern of eight bits; notice the repeated 
pattern in a bitmap which uses patterns of black and white bits to simulate a shade of 
grey; and realize that blocks of pixels which are repeated may also be considered a mn 
of patterns. 

Regardless of the type of data (ASCII characters, binary data, etc.), mn-length 
encoding is guaranteed to reduce the physical size of the file if the length of each mn 
is greater than the number of bytes or bits substituted for the original sequence in the 
compressed file. 

A. TERMINOLOGY 

It is appropriate at this point to define the terminology which is used in this 
section. As will be explained in greater detail, a mn is encoded by the following 
elements: 

• compression indicator character: any seldom-used predefined character or bit 

pattern which indicates that compressed data follows. 

• run value; a bit, a pixel, a character, or a pattern which is repeated. 

• run length: the number of times the mn value is repeated. 

An important concept which will be used repeatedly is that of a minimum run 
length. This is the minimum number of consecutive values that must be in a mn for 
mn-length encoding to be beneficial. In other words, it is the break even point 
between an increase in file size caused by the three elements just mentioned, and a 
savings in file size made possible by compression. 
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B. COMPRESSINC DATA FILES USING NULL SUPPRESSION 

Null or hinnk suppression is a simple type of nin-length encoding, and 
represents one of the earliest uses of data compression. It is particularly useful on files 
which contain fixed-length records, such as language source programs and database 
files. Null suppression reduces rep>eated occurrences of the null or blank character to 
two bytes. One byte contains a compression indicator character, usually an unprintable 
character or seldom-used character chosen from the character set (ASCII, EBCDIC, 
etc.) used by the file. If the compression indicator character does appear in the 
original data file, it can be made unambiguous by doubling its appearance in the 
compressed file. The second byte contains the number of null characters in the run. 
The upper limit of 255 (the maximum value which one byte can represent) is adequate 
for a text data file. 

Figure 3.1 illustrates a simple file compressed by blank suppression. Notice that 



(A) 


Original data file: 
(fixed length records) 




N AN C Y D RE W ^ 
ADVENTURE LANE 
MY STERY 

'^Y ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 


(B) 


Compressed data file: 




NANCY -'DREW0 ( 1 0 ) ADVENTURE ''LANE 0 (6)MYSTERY 
0 (13)NY0 (18) 



Figure 3.1. Compression by Blank Suppression. 



the numbers in parentheses are decimal representations of the bit configuration of the 
run length and only occupy one byte. The character is used to represent the blank 
character. 
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The original file requires 80 bytes, whereas the compressed file contains only 41 
bytes. This is a compression ratio of nearly 2:1, or 49%. It is perhaps more 
significant, however, to only consider the compression on the blank characters of the 
file; in this case, 50 blank characters were replaced by ten bytes of encoded data, 
producing a compression ratio of 5:1, or 80%. It is clear from this example that no 
advantage is gained from compressing mns of fewer than three blanks. Therefore, the 
minimum run length for null or blank suppression is three characters. 

Example . NARC.EXE is a public domain shareware product for microcomputers 
written by Gary Conway. It is a de-archiving facility for storing files in a compressed 
format. Several storage methods are available to allow the user to "pack," "squeeze," 
"crunch," or "squash" a file. "Packing" is an implementation of blank suppression. 
Conway says that 

[Packing] is the simplest of the storage methods. Suppose that you have a 
line of text and at the end of the line, you have 40 spaces. These 40 spaces are 
compressed into 3 bytes in the ARC* file. The first byte is the actual character 
to be expanded (in our case a space). The second byte is a special "flag" byte 
that indicates that we need to expand these bytes. The third byte is the count 
byte (in our case it would be 40). So you can see that any time the ARC’er 
finds repeated bytes like this, it can compress them into 3 bytes. [Con87: p.9] 

Notice that the NARC method requires three bytes to compress a run, compared to the 

two bytes described previously. While the NARC technique has the disadvantage of 

taking more space and not gaining the maximum compaction available through null 

suppression, it has the advantage of being more general, as does the following method. 

C. COMPRESSING ASCII DATA FILES 

To use run-length encoding on an ASCII data file requires not two, but three, 
bytes of information: one byte for the compression indicator character, another byte 
for the run length, and a third byte for the value of the character being repeated. For 
this type of application, the minimum run length is four characters and the maximum 
run length is 255 bytes. Also, compression might be feasible if a fDe of 
alphanumerical data contained patterns of characters, such as a repeating number. 
However it appears that run-length encoding is not a good candidate for compressing 
an English text file other than through null or blank suppression. 
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n. COMPRESSING EIGHT-BIT GRAPHICS FILES 



Run-length encoding is particularly beneficial when the information represented 
is a bitmapped graphics file. Let us first look at a type of graphics file which contains 
eight-bit codes to determine the feasibility of run-length encoding. This is the grey- 
scale or eight-bit color graphics data file; each pixel in the bitmap is represented by 
eight bits, or one byte. Again, the minimum run length is four pixels, but finding four 
or more consecutive pixels of the same shade or color is highly probable. 

Refer to Figure 3.2 for an example of this implementation. In this figure, the 



(A) 


Original bitmap: 




BBBBBBBBBBBBBBBBBBBB 

wwwwwRwwwwwwwGwwwwww 

wwwwRRRwwwwwGGGwwwww 

wwwRRRRRwwwGGGGGwwww 

wwwwRRRwwwGGGGGGGwww 

wwwwwRwwwwwwwBwwwwww 

BBBBBBBBBBBBBBBBBBBB 


(B) 


Compressed data: 




0 (20) B0 (5) wR0 (7) wG0 (10) wRRR0 (5) wGGG08w0 (5) Rwww 
0 (5)G0 (8)wRRRwww0 (7)G0 (8)wR0 (7)wB0 (6)w0 (20)B 


Figure 3.2. 


Compression of Graphics Data. 



small bitmap (A) shows a red diamond and a green tree on a background of white with 
black borders. The original bitmap takes 140 bytes. The compressed file (B) uses 61 
bytes. This is greater than 56% compression. 

Again, the numbers in parentheses are decimal representations of the run length 
and only occupy one byte. The run length contains values from four to 255 pixels 
because a minimum run length of four pixels indicates that no compression wUl occur 
on runs of length zero through three. Conceivably this byte could be used to contain 
run lengths from four to 259 pixels, thereby increasing the maximum run length coded 
in one byte. This technique of increasing the maximum capacity of a byte in this 
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manner may be applied to other examples as well. If a mn is greater than the 
maximum permitted, it may be expressed as several runs of the same value. For 
instance, a run of 400 red pixels may be compressed into the following six bytes: 
@(259)R<a)(141)R. 

Example . GEM“, a software product of Digital Research Inc., stands for 
Graphics Environment Manager. It offers mn-length encoding as one option for 
compressing a bitmapped file where each pixel occupies a variable number of bits, say 
eight. In this particular situation, encoding is implemented as follows. 

A "mn-length packet" is used to represent a mn of less than 128 pixels. This 
packet consists of two bytes of information: the length of the mn and the eight-bit 
value of the pixel. Since a bitmap is logically one long string of information, a single 
mn of pixels may be longer than a line on a monitor. 

For a mn greater than or equal to 128 pixels, an "extended mn-length packet" is 
used. This is a three-byte packet containing the following information: an opcode of 
value -1, an extended mn byte containing a count of 128-pixel mns, and the pixel 
value. Figure 3.3 illustrates this concept. 



(A) 



Normal 
Byte 1 
Byte 2 



run-length packet (run <128 characters): 

Run length 
Run value 



(0-127) 



(0-255) 



(B) 



Extended run-length packet (run >=128 characters) : 



Byte 1 
Byte 2 
Byte 3 



1111 1111 
(0-255) 
(0-255) 



Opcode 

Long-run multiplier 
Run value 



Figure 3.3. Run-length Packets in GEM“" . 
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Figure 3.4 shows an example of a run consisting of 1000 pixels in length. An 




efficient way to encode this is to use an extended run of length seven (7 * 128 = 896 
pixels) followed by a standard run of length 104. [DRI85: p.I-2] 

For completeness, it should be mentioned that under this implementation the 
entire file is encoded. However the encoding method also includes options other than 
run-length encoding, such as pattern encoding which is indicated by an opcode value 
of -3. 

E. COMPRESSING BINARY IMAGE FILES 

Some interesting variations of run-length encoding exist for a binary image file. 
If graphics data contains only black and white values, then each pixel can be 
represented by a single bit. Compressing such a file using normal run-length encoding 
as previously described, is not the best method. 

For instance, in our previous example, a pixel was represented by one byte of 
data; a minimum mn length of four pixels or less provided reasonable compaction. 
With binary data, using the same implementation yields a large minimum run length. 
The three bytes required to compact one run could hold 24 pixels. Thus, in order to 
benefit from compaction, a run must be at least 25 pixels long. 

Depending on the size of the bitmap and the degree of uniformity of the graph, 
.such an implementation may render acceptable compression. Tf. for instance, a bitmap 
of 1024 by 1024 pixels contains mostly white space, or long, horizontal black lines, 
the long mns would qualify for compaction. But variations of run-length encoding 
offer greater efficiency for binary image data. 
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The first decision to make when considering compression of a file is "to 
compress or not to compress." Unless a file contains a high percentage of nins which 
exceed the minimum mn length, the human and computer resources required may not 
be worth the effort. 

If we can decrease the number of bytes required to compress a run, and hence 
reduce the minimum run length, then a file may more easily qualify for compression. 
There are several techniques for accomplishing this objective. 

Method One . One technique is to encode the entire data file, regardless of the 
length of a run. This decision immediately eliminates the need for the compression 
indicator character, and reduces the minimum run length from 25 pixels to 17 pixels. 

In addition to deleting the indicator byte, if we then combine the length of a run 
and its value into one byte, we have further reduced the minimum mn length to nine 
pixels. With this encoding scheme one byte of compressed information would appear 
as follows: 

bit one: value ("0" or "1") 

bits 2-8: mn length (0 - 127) 

Runs longer than 127 would simply require two or more bytes to represent them. 

Method Two . Alternatively, by using the entire eight bits of a byte of 
compressed data, we could represent a maximum run length of 255 pixels. Since the 
entire binary file is being encoded, why not simply transmit a series of mn lengths? 
If we make certain assumptions about the compressed file, then this is possible. Let 
us assume that the first value transmitted is always zero ("0"). Let us also assume that 
for a mn greater than 255 of a particular value, a "null" byte of ran length zero for 
the opposite value will be inteijected between bytes showing the longer mn. While 
this method achieves better compression on mns of lengths 128 to 255 bits, the 
overhead incurred by the null byte for mns greater than 255 bits reduces the efficiency 
of the method. 

Method Three . In order to make maximum use of each byte for compressing 
data, and yet assume transmission of alternating values, beginning with a "0" bit, we 
add another modification. To handle the problem with long mns. implement a "base 
127" approach. Runs of length zero to 127 bits are represented by a single byte 
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containing that value. If a run exceeds 127 bits, then use values from 128 through 255 
to represent a multiple of 128. Notice that for these values the leftmost bit of the 
compression byte is "1." This bit turned on signifies that two bytes, rather than one, 
contain the run-length value. This concept is similar to that used in GEM® in the 
example above. This variation on run-length encoding is also the compression 
technique used by Michael Gunning in the GRAFPC program which wUl be examined 
in Chapter Vn. 

Figure 3.5 compares Methods One, Two, and Three, showing the different number 
of bytes required to compress a sample file. In this figure, the numbers in parentheses 
represent the run length contained in one compression byte. In Method One, the run 
length value is shown as the first bit of the compression byte. In Methods Two and 
Three, a null byte is transmitted first since the assumption is that a compressed bit 
stream begins with a zero value. 

In conclusion, we can infer from the example that, as the length of the runs 
increases, so will the number of bytes required by Methods One and Two, but Method 
Three wUl never require more than two bytes to represent a run. However, all three 
methods give excellent compression. 

F. ENCODING PATTERNS 

Another important variation on normal run-length encoding considers a mn of any 
pattern to be a candidate for compression. One possible method for encoding a 
pattern requires that four items of information be used for each run; 

• A compression indicator character to indicate the start of a run 

• The length of the run 

• The length of the pattern 

• The pattern 

The pattern length must be included because it is variable. 

This approach will work for an ASCII data file which is inspected byte by byte. 
However, it is a more complicated problem to encode a raw bitmapped data file, 
where a string of bits may appear in any conceivable configuration. For instance, 
consider a dithered bitmap which sunulates grey-scale filled area with runs of the 
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Run 


Run 


Method 


Method 


Method 


Value 


Length 


One 


Two 


Three 


"O" 




( 0) 




( 0) 


II ^ 11 


50 


( 50) 


K 50) 


( 50) 


II QH 


100 


<100) 


0 (100) 


(100) 


II 2^ II 


200 


(200) 


1 (127) 


(128) 








K 73) 


( 72) 


II on 


300 


(255) 


0 (127) 


(256) 






( 0) * 


0 (127) 


( 44) 






( 45) 


0 ( 46) 




II 2^ II 


400 


(255) 


1 (127) 


(384) 






( 0) * 


1 (127) 


( 16) 






(145) 


1 (127) 
1 ( 19) 




II on 


500 


(255) 


0 (127) 


(384) 






( 0) * 


0 (127) 


(116) 






(245) 


0(127) 

0(119) 




II 2^ II 


600 


(255) 


1 (127) 


(512) 






( 0)* 


1 (127) 


( 88) 






(255) 


1 (127) 








( 0) * 


1 (127) 








( 90) 


K 92) 




Number of bytes to 








compress 2150 


bits : 


18 


20 


13 


Compression : 




93.3% 


92.6% 


95.2% 


* Indicates a 


Null byte of 


the opposite 


value . 



Figure 3.5. Compression of a Binary Lnage File Using Run-Length Encoding. 



repeated pattern "on-on-off." In such a file, how can we embed, and expect to 
recognize, a unique indicator character? 
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One method is to define an arbitrary bit pattern to represent an indicator 
character, e.g., "11111111." In order to distinguish this indicator from the appearance 
of the same string in the raw data, the protocol of doubling an original occurrence is 
used. In other words, if "11111111" appears in the original bitmap, aligned on a byte 
boundary, then two identical bytes of "11111111" are transmitted. Next the file is 
processed byte by byte, searching for an indicator pattern. If not found, the byte is 
transmitted as raw data; if found, the next byte is checked to determine if this is an 
indicator or raw data. If this byte is the indicator, then the following data is treated 
as encoded data. A second indicator character indicates the end of compression. 
[May88] 
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IV. STATISTICAL CODING 



Run-Length encoding generally requires that a fixed number of bits or bytes be 
used to represent a series, or run, in the file to be compressed. Now we consider a 
situation where variable-length codes may be used. The main idea inherent in 
statistical coding is that if some symbols occur more frequently than others in a 
message, then we can take advantage of this fact by using a variable-length code 
such that frequent symbols will be replaced by shorter codes, while symbols 
which are used less frequently will be replaced by longer codes. This concept is 
used in the Morse coding system, for instance. The most common letter in the 
alphabet, E, is represented by a single "dot," whereas a Q is transmitted as "dash-dash- 
dot-dash." 

A. TERMINOLOGY 

In this discussion of statistical coding, the following terminology will be used to 
identify the data to be compressed. The entire file to be compressed is referred to as 
the message. Once encoded for transmission, the message becomes the encoded 
message. 

The information elements of a message are referred to as symbols. A symbol 
may be a character of the alphabet, it may be a bit representation of picture elements 
(pixels) in a graphics file, or it may even be a group of characters or bits. The set 
of symbols used in a message is referred to as S,, Sj, Sj, ... S„, where n is the 
cardinality of the set. Each symbol, Sj, has associated with it both a length, Lj, and 
a probability, Pj. The length is the number of bits used to encode the symbol, whereas 
the probability indicates how often the symbol is used in the message. 

The .source alphabet of a file is the set of all possible symbols which may be 
used in a message. Examples include the set of ASCII characters, the lower-case 
English alphabet, and the set of all colors that can be represented by eight bits. 
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Source symbols are a subset of the source alphabet; these are the symbols which 
are actually used in the message. One example would be all the colors present in a 
graphic image: 58, say, as opposed to the possible 256 colors available. 

Code symbols are the variable-length strings assigned to encode the source 
symbols; these are generally listed in a decoding table. The elements, or coding 
digits, which are used to comjX)se a code symbol are "0" and "1" in the binary number 
system, which has a radix of two. However systems of a radix other than two are 
used; for instance, the Morse codes use coding digits from the set {"dot”, "dash", 
"pause"} which has a radix of three [Ham86: p.l5]. 

Lastly, a code may be thought of as a mapping of source symbols onto code 
symbols. The decoding table shown in Figure 4.1. contains a sample code. The act 
of exchanging the set of source symbols which make up the message into the set of 
code symbols which compose the encoded message is referred to as encoding. A 
simUar process, in reverse, will generally decode the encoded message back to the 
original message. 



Source symbols; 

A B C B D A A — > 



encoder 



Code symbols: 

— > 1010010100011 



Decodina Table: 

A 1 

B 01 

C 001 

D 000 



Figure 4.1. Source Symbol to Code symbol. 



B. CODING AND INFORMATION THEORY 

The presentation of some background in the coding theory which underlies 
statistical compre.ssion methods will provide us with a measure for evaluating the 
effectiveness of these methods in general, and Huffman’s method in particular. If we 
view information theory, as Lelewer suggests, as the study of efficient coding and its 
consequences on the speed of transmission and probability of error, then we perceive 
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the primary objective of data compression as minimizing the amount of data to be 
transmitted. [Lel88: p.261] 

1. Differentiation of Codes 

The following terms are used to differentiate among codes. Also several 
properties exist which should be incorporated into the design of a code. 

First, it is desirable to be able to distinguish one code symbol from another. 
In other words, we want to establish a one-to-one correspondence in mapping the set 
of source symbols onto the set of code symbols. Such a code is called distinct. 

One problem in using variable-length codes is how to determine the end of 
one code and the beginning of another. A code is uniquely decodable if it is distinct 
and each code symbol embedded in an encoded message can be readily identified. A 
code in this category produces an encoded message which can have only one possible 
interpretation. In order to use variable-length codes, the code must have unique 
decodabUity. To use shorter codes for symbols with a high probability implies the use 
of variable-length codes. 

But it is also desirable to know immediately when a complete symbol has 
been received, without having to examine other transmitted symbols before deciding 
what code symbol was sent. Morse codes use a "pause" of a predefined length as a 
code delimiter to meet this requirement. Another method is to insure that the code 
selected has the prefix property, which assures that no encoded symbol of this code 
is a prefix of any other symbol. If a code that is uniquely decodable has the prefix 
property, it is said to be instantaneously decodable and is referred to as a prefix code 
or an instaneous code. [Ham86: pp.52-56][Lel88: p.264] 

Figure 4.2 illustrates the inherent hierarchical structure of the codes 
described. Figure 4.3 shows examples of codes which fall into the categories identified 
in Figure 4.2. 

2. A Finite Automaton 

One tool for determining instantaneous decodabUity is a finite automaton, 
which can also be represented as a decision tree, or a "decoding tree." This concept 
wUl become clearer through the use of an example. 
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instantaneous 

(prefix) 

codes 





uniquely 

decodable 

codes (B) 



distinct 

codes (A) 



Figure 4.2. 


Hierarchy of Codes. 








(A) 




(B) 


(C) 




Distinct 


Uniquely 


Decodable 


Instantaneous 


Si = 


0 




0 


0 


S2 = 


01 




01 


10 


S3 = 


11 




Oil 


110 


S4 = 


00 




111 


111 



Figure 4.3. Examples of Codes of Different Categories. 



Suppose that our message contains only four symbols, S,, Sj, Sj and S 4 , and 
that each symbol is represented by some binary code, i.e., the coding digits are " 0 " and 
" 1 ." Assume the following assignments are made. 

S, = 0 
S, = 10 
S, = 110 

S4 = 111 

Notice that the prefix property is preserved in these assignments since no code symbol 
is the prefix of another code symbol. 
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Next consider the finite automaton representation of this assignment in 
Figure 4.4 fHam 86 ; p.54]. Each node (or vertex) represents a state and each arc (or 
edge) a transition between the nodes it connects. Every arc is labeled with "0" or "1," 
depending on whether it is a left-handed or right-handed branch. It is arbitrary whether 
all branches in a particular direction are denoted by "0" or "1." The code for any 
particular symbol is obtained by following the path from the start state to the state 
where that symbol is defined, and by recording the labels of the arcs encountered along 
the way. If following the path of a code does not lead to a final, or "accepting" state, 
then the code does not preserve the prefix property. 



0 

# 

0 


s. 


= 0 


Q — 


1 

S, 


= 10 


\ ^ 


S, 


= no 


0 


s. 


= 111 


0 

(S) 


4 





Figure 4.4. Finite Automaton. 



Given the above explanation, one may wonder why the following finite 
automaton, or decision tree, would not work as well. This tree also preserves the 
prefix property. Both trees illustrate codes which are instaneous. 

While it is true that both trees meet the prefix criterion, we must ask which 
symbol encoding will produce the better performance? The answer lies in knowledge 
of the statistical makeup of the message. We must answer the question "What is the 
probability of occurrence of each symbol in the message?" If the frequency of 
occurrence is evenly distributed, that is, all symbols occur with equal probability (like 
tossing a coin), then the tree in Figure 4.5 is better. But if S, occurs more frequently 
than Sj or S 4 , then perhaps we can capitalize on the fact that more of the message will 
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0 






S. 


= 00 




s. 


= 01 


(S; 


2 




S3 


= 10 


(S,) 


S. 


= 11 



(S,) 



Figure 4.5. Binary Tree. 



be represented by a one-bit code than by a three-bit code, thus producing a higher 
degree of compression and a shorter encoded file for transmission. Later examples of 
Huffman code implementation will demonstrate this fact. 

3. Noiseless Coding Problem 

Statistical compression techniques are generally approximations to the 
solution of the noiseless coding problem. Assuming a noiseless channel allows for a 
system in which the code symbols can be transmitted from one point to another 
without possibility of error [Lel88: p.262]. The goal is to construct a uniquely 
decipherable code which also minimizes the average length of the code symbols, where 
the average length is a function of the probability and length of each symbol. The 
formula is expressed as L.,, = Z PiL,. Such a code is referred to as an opliimiin code. 
The ability to produce an optimum code is valuable because the shorter the length of 
the code symbols, on the average, the shorter the message, and therefore the shorter 
the time required to transmit the message, which is our ultimate goal in compressing 
data for transmission. 

From the two decoding trees above, we will con.stmct an example to use 
throughout this section. fDav88] Given the four symbols from Figure 4.4, we assign 
arbitrary probabilities of occurrence to each symbol, and do likewise for the symbols 
from Figure 4.5. In Figure 4.6, situation (A) shows that the symbols occur with 
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unequal probability, but for situation (B) all symbols occur with equal probability. Tbe 
average code-symbol lengths are calculated for each circumstance. 




We can see that the two codes have very different average code-symbol 
lengths with situation (A) being smaller than that in (B). But how do we know that 
the code in (A) is an optimal code? 

4. Entropy 

The degree of optimality of a code can be measured by the entropy of the 
message [Aro77; p.l7]. The formula for calculating entropy is 

H = -lP<log,P, = Z P, log, (1/P,). 

This value provides a lower bound on the average code length for an optimal code. 
In other words, a code may have an average code-symbol length very close to, but not 
less than the entropy. If the average length of the code symbols compares favorably 
to the entropy value, then the code is considered to be optimal. 

Then what is entropy? Webster’s Dictionary defines entropy as "a measure 
of the amount of information in a message that is based on the logarithm of the 
number of possible equivalent messages." Hamming describes entropy as "the average 
information of the [source] alphabet." [Ham86; p.l08] He states. "The entropy function 
measures the average amount of uncertainty, surprise, or information that we get from 
the outcome of some situation, say the reception of a message..." [Ham86: p.ll4] 
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For instance, if the probability of a source symbol, say S,, is equal to one, 
then the probabilities of the other symbols in the alphabet equal zero. TTierefore, since 
from all possible symbols in the source alphabet it is certain that the next symbol we 
receive is Sj, then there is no surprise about the transmitted message, and hence no 
information; the entropy is -logjl, or zero. 

If, however, the source symbols have a probability distribution as in Figure 
4.6 above, the transmitted message is uncertain and does contain information. 
Calculating the entropy of examples (A) and (B) we get values of 1.75 and 2.0, 
respectively. This is because in these examples the length of the symbol happens to 
equal the logj of the probability of the symbol. The fact that the calculated values for 
L„g and H are the same illustrates the high degree of optimality of these particular 
codes. 

Consider a similar code in Figure 4.7 where this condition does not hold. 




The entropy (H) for this example is calculated as 1 .b.*' and the average code-symbol 
length (L,^) is 1.70. TTiis code is not as optimal as tho.se in Figure 4.6. 
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C. HUFFMAN CODES 



In 1952, Dr. David Huffman, published a paper entitled "A Method for the 
Construction of Minimum-Redundancy Codes." [Huf52] Building on earlier work by 
C. E. Shannon [Sha48] and R. M. Fano [Fan49], Huffman produced a method to derive 
an optimum code which results in the shortest average code length of all statistical 
encoding techniques [Hel87: p.97]. In his paper Hufftnan defines five basic restrictions 
which an optimum code must meet. They are quoted below. The reader should be 
aware that Huffman’s terminology differs somewhat fi"om that presented earlier. He 
uses the term "message" or "message code" for "source symbol." 

(a) No two messages will consist of identical arrangements of coding digits. 

(b) The message codes will be constructed in such a way that no additional 
indication is necessary to specify where a message code begins and ends 
once the starting point of a sequence of messages is known. 

(c) L(l) <= L(2)<= ... L(N-1) = L(N). 

(d) At least two and not more than D of the messages with code length L(N) 
have codes which are alike except for their final digits. 

(e) Each possible sequence of L(N)-1 digits must be used either as a message 
code or must have one of its prefixes used as a message code.^ 

Restriction (b) is the requirement which demands "unique decodability." Restriction 

(c) assumes that messages are ordered with the probability decreasing and the length 

of the code for the message increasing. This restriction states that in order to have 

an optimum code, it is necessary that the length of the last symbol in the list (the one 

with the lowest probability of occurrence) equals the length of the next-to-last symbol. 

In restriction (d), "D" is the radix, that is, the number of coding digits available for 

encoding a symbol. For a binary coding system such as that used in the examples of 

this thesis, D always equals two. 



^ After constructing numerous Huffman trees, this author is of the opinion that restriction 
(e) as stated by Huffman is incomplete. The author believes it should be altered as follows: 
(e) Each possible sequence of L(N)-1 digits must (1) be used as a message code, (2) have 
one of its prefixes used as a message code, or (3) must itself be the prefix of another 
message code. 

Indeed, one and only one sequence of digits will be the prefix of all of the last D message 
codes in the list, as established by restriction (d). 
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1. Description of Technique 

Having stated these restrictions, let us now look at the creation of a 
Huffman coding scheme. The process consists of the following steps: 

• Define the frequency distribution of source symbols. 

• Construct a Huffman code for the source symbols. 

• Encode the message. 

• Transmit the message. 

• Decode the message. 

Step One . The first step is to define a frequency distribution; this entails 
assigning a probability of occurrence to each symbol used in the message. One 
method of doing this is to scan the input file, tabulating data for each symbol and 
building a table of probabilities in the process. This table will be used in Step Two, 
refined, and transmitted with the encoded file to be used in the decoding process. 

Another method of obtaining frequency information is to use a predefined 
table which was developed for a source alphabet similar to that used in the message. 
For instance, tables already exist which define usual probabilities of occurrence for 
each letter in the English alphabet. This table would suffice for large text files. A 
different table would be required if the symbols were words in a programming 
language. 

Step Two . Constructing a Huffman code from source symbols is the next 
step. Our objective is to build a tree in which the nodes represent probabilities. The 
leaf nodes will be labelled with the original probabilities associated with each source 
symbol in the message, internal nodes will consist of derived probabilities as described 
in the following steps, and the root node will have the unity probability of 1.00. The 
arcs of the tree are each labeled with one coding digit, either "0" or "1." Theoretically 
the labelling is arbitrary, but for our example, let us label all branches toward the top 
as "0" and branches toward the bottom as "1." Reading the path from the root node 
to a leaf node will provide the Huffman code for each symbol. Adhering to the 
restrictions imposed above, we next describe the technique for building a Huffman 
code: 
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• Arrange the symbols by decreasing probability, with the symbols least likely to 
appear in the message at the bottom of the list. Restriction (c) above states that 
the two least frequent source symbols have the same encoded length. We begin 
with the probabilities of these two symbols. 

• Form a new node by summing the probabilities of the two nodes at the bottom 
of the list. The two nodes combined will always be the two which have the 
lower probabilities on the current list. 

• Make a new list of ordered probabilities with the new node inserted into a proper 
position in this list. Note that if one or more nodes already exist which have the 
same probability as the new node, it does not matter whether the new node is 
placed before or after the existing node(s). Hufftnan states that "it is possible to 
rearrange codes in any marmer among equally likely messages without affecting 
the average code length." [Huf52: p.llOO] 

• Repeatedly perform the two previous steps until the list contains only two nodes. 
The sum of these two probabilities will equal one; this is the root node and we 
have built the desired encoding tree. Refer to Figure 4.8. It can be seen that 
for a source alphabet containing n symbols, the resulting tree will contain n 
levels, including the root node. 




• Convert the diagram shown above to a tree structure, maintaining the relative 
location of the arcs such that of the two arcs entering a node, the upper arc is 
drawn from the node having the greater probability and the lower arc comes from 
the node having the lesser probability. If the two nodes which are combined to 
form a new node have equal probability, then their relative location does not 
matter. Remember that each time a new node is formed, it is always the current 
two lower probabilities that are joined. The tree diagram shown in Figure 4.4 
shows the tree structure derived ft’om the diagram in Figure 4.8. This example 
produces a simple tree since no transpositions of probabilities occur. Alternate 
visual representations by Held [Hel87: p.97] and Davis fDav88] are shown in 
Figure 4.9, (A) and (B), respectively. 
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Figure 4.9. 



Derivations of Huffman Codes (Held and Davis Diagrams). 



• Assign a "0" to each upper arc of the tree, and a "1" to each lower arc. 

• Beginning at the root node, follow each path back to a leaf node, recording the 
label of each arc encountered along the way. Each bit string thus derived is the 
encoded value for that particular source symbol. For instance, following the path 
to symbol C we record the labels "1," "1," and "0;" thus the encoded value of 
symbol C is the three-bit string "110." 

Step 3 . The next step in the process is to encode the message, using the 
codes for the source symbols derived in Step Two. This is a straightforward 
substitution encoding process, producing a compressed file as output. For example, 
source symbols "ABBADAAC" would be encoded as code symbols "0101001 11001 10". 

Step 4 . Next the encoded message is transmitted. If a table of codes was 
derived as just described, then this information must also he sent with the message. 
However, if a predefined frequency distribution of probabilities for the source alphabet 
was used in the coding process, then the receiver may either re-create the Huffman 
codes on the receiving computer, or may already have a Huffrnan coding table resident 
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on that computer. When the table must be passed with the message, the effect on the 
size of the transmission due to the table size must be taken into account in determining 
a valid compression ratio. 

Step 5 . The final step is to decode the encoded message. It is in the 
decoding process that the concept of "unique decodabUity" is fully appreciated. As 
each variable-length code is received, it can be conclusively associated with one and 
only one code from the decoding table. 

Thus decoding is easily accomplished in one or two passes, depending on 
the need to derive a decoding table. 

2. Another Huffman Code Example 

Now that we understand the process of deriving Huffman codes, let us look 
at a more complicated example. Assume that the data file, i.e., the message, is a color 
graphics bitmap, and each pixel is represented by three bits. Scanning the data produces 
a frequency distribution of the colors. Of the eight available colors (the source 
alphabet), only seven are actually used (source symbols). Figure 4.10 shows the source 
symbols, the probabilities of occurrence, and the list reordering process. Notice the 



Si 

white 
red 
green 
blue 
black 
cyan 
yellow 

Figure 4.10. Huffman Code Example. 

transpositions which occur during this process. 

Next, using Held’s visual representation of a tree stnicture, we construct the 
tree from which to read the code symbols. This construction is shown in Figure 4.1 1. 

The minimum average code length can be calculated for this example from 
the formula X Pib = 2.40. We compare this to the calculated value for entropy to 
estimate the optimality of our derived code X -Pilog.Pj = 1 79. 
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Figure 4.11. Huffman Code Example (Tree Diagram). 



Without compression the message requires 3 bits per symbol; with 
compression it requires an average of 2.4 bits per symbol, a savings of .6 bits f>er 
symbol. If we consider, for example, an EGA bitmap with a resolution of 640 by 350 
pixels, then the compressed file is 134,400 bits versus 224,000 bits, a savings of 20%. 
3. Dynamic Huffman Codes 

The Huffman codes which have been discussed so far are those which are 
either developed by assessing the frequency of the elements used in the message, or 
chosen from existing tables. Such tables are static, predetermining the probability, and 
thus the order, of the source symbols. Both the sender and the receiver have on hand 
identical tables prior to the transmission of the actual message. 

It is also possible to dynamically develop Huffman tables. A good 
description of dynamic Huffman coding is given in the following quotation. 

In a dynamic Huffman model, a frequency algorithm determines which 
characters are represented at which levels in the table. Every time a character 
is used, its position in the table is exchanged for the position of the character 
immediately above it. The bit patterns in the table themselves do not actually 
change. WTiat changes is the assignment of the bit patterns within a table entry 
to represent a particular character. An exchange is always made after the code 
currently assigned is sent across the line. This ensures that both sender and 
receiver can update their respective copies of the table in synch. [Bac88: 
p.77,78] 

An example of a situation where it would he advantageous to u.se the 
dynamic-table method is a text data file which contains lengthy sections of all 
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uppercase characters interspersed with uppercase and lowercase text. The frequency 
distribution of the normal text would probably give all uppercase letters very low 
probabilities. However, in the switch to uppercase letters only, it would be desirable 
to represent these characters with the shorter codes reserved for frequently used 
characters. Developing a new table dynamically would allow the uppercase letters to 
rise to the top of the table, and to be encoded in a shorter form. 
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V. RELATIVE ENCODING 



Numerous other lossless compression techniques exist, including the Lempel-Ziv 
method, predictive encoding, adaptive encoding, and relative encoding. Many methods 
are a combination or modification of techniques already mentioned. Telcor 
Corporation, for instance, claims to have developed a method which combines elements 
of Huffman and Lempel-Ziv methods, but produces greater compression than either 
[Bac 88 : p.77]. 

A. DESCRIPTION OF RELATIVE ENCODING 

We shall examine one of these methods, the relative encoding technique, which 
is used to transmit data over facsimile devices. This is of interest because of the 

similarities of this type of data to a graphics bitmapped image. Facsimile data 

typically is transmitted as 1728 points or pixels per scan line with approximately 850 
scan lines for a standard 8 V 2 by 1 1 inch page. 

Relative encoding takes advantage of the fact that facsimile images generally 
contain a much higher quantity of white space than black space. There may be little 
change from one scan line to the next. This method transmits only the difference 
between scan lines. The process of encoding a file by relative encoding is described. 

• Read the first scan line into a buffer in memory. Transmit this line exactly. 

• Read the next scan line of the file into a second memory buffer. Compare this 

line to that in the first memory buffer. Transmit only the location of the pixel 
where a change occurs. 

• Move the scan line in the second buffer to the first buffer. 

• Continue to execute the two previous steps until all the scan lines of the file have 
been compared and differences have been transmitted. 

This process will become clearer with an example. The asterisks represent the 

positions where a change has occurred from the top line to the next line. 

00001 1 1 1 100100000000.. .001 1 1 Nth scan line 

00001 1 1 100001 1 1 10000.. . 00001 (N+l)th scan line 

* ***** ** Relative change 
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R. HOW TO DESIGNATE RELATIVE CHANGE 



There are two ways to designate the relative change or the difference shown from 
one scan line to the next: "positional notation" or "displacement notation." 

1. Positional Notation 

A positional indicator may be used for each relative change to indicate the 
location of the pixel where a change occurs; the location is the number of the pixel 
changed, relative to the first pixel of the scan line. In our example above, the 
transmission for the (N+l)th scan line would indicate only the locations of the changed 
data, i.e., 9, 12, 13, 14, 15, 16 1726, 1727. 

If each digit in a scan-line sequence is transmitted in a four-bit nibble, i.e., 
digits are packed two to a byte, then greater compaction results than from using a byte 
to contain the same data. One alternate technique of positional notation allows for 
the transmission of a positional indicator plus a count of the number of successive 
changes. Using this method, the above example would result in the transmission of 
these values: 9, 1, 12, 5, ..., 1726, 2, which further increases compaction. 

2. Displacement Notation 

Another method for reducing the number of digits required to indicate 
change is to employ displacement notation. Because a facsimile scan line is long 
(1728 pixels), changes near the end of a line require four digits as opposed to one or 
two at the beginning of the scan line. To alleviate this end-of-line increase of digits, 
the actual location of the first change of a scan line is transmitted, but successive 
changes in the same line are transmitted as a displacement from the previous change 
location. Assuming that the change previous to that shown for location 1726 was at 
location 1716, the above example would be transmitted as 9, 3, 1, 1, 1, 1, ..., 10, 1. 

The alternate method of representing changes which are adjacent by a count 
of the successive changes applies for displacement notation as well. The above 
example translates to a transmission of 9, 1, 3, 5, ..., 10, 2. 
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VI. EVALUATION 



We have discussed techniques for compressing data using run-length encoding, 
statistical coding, and relative encoding. Since our interest is primarily in the 
effectiveness of these methods for compressing graphical bitmapped data, we shall 
analyze each in terms of (a) binary (black and white) images requiring one bit per 
pixel, and (b) grey-scale and color graphics images, comprising more than one bit of 
information per pixel. Binary graphics images, which are analyzed on a bit-by-bit 
basis, sometimes require a different type of processing from grey-scale and color 
images, which are analyzed one or more bytes at a time. 

A. RUN-LENCTH ENCODING 

Run-length encoding can be used on any type of graphical image. The key 
questions to ask are: 

• Is there enough repetition in the file to warrant encoding? 

• What is the minimum run length for this file? 

Remember that repetition may refer to runs of identical pixels or runs of repeated 
patterns of pixels. Also recall that typically the minimum run length decreases as the 
number of bits per pixel increases. 

The run-length encoding of a binary image file is most efficiently performed by 
using a method described in the previous chapter, i.e., by encoding the entire file where 
each run is represented by one byte, such as an ASCII character selected from a 
lookup table. But even though each pixel in a binary image file occupies only one bit, 
encoding a run may require eight or more bits, with a minimum run length of nine 
pixels. By comparison, an eight-bit grey scale pixel may be encoded in two bytes, 
with a minimum run length of three pixels. 

What are the best, worst, and average amounts of compression attainable by run- 
length encoding a bitmapped graphics file? 

For all bitmapped files, the best compression is on a file which has only one 
value, and is thus composed of one very long run. The worst compression would be 
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found in a file which contains no runs of length greater than or equal to the minimum 
run length. Compression is 0.0% since to encode such a file would create an encoded 
message larger than the original message. And the average case could fall anywhere 
in between the extremes. But with any type of mn-length encoding, the more 
repetition the data contains, the more successful the encoding; the longer the runs, the 
greater the compression. 

1. Best Case — Binary 

For a binary bitmapped file, the method of compression determines the 
maximum run length that can be carried in one byte, and thus has an effect on the 
maximum degree of compression attainable. In Chapter II, were described three 
methods for compressing this type of file. Each of these methods assumes that the 
entire file is encoded, as in the best case examined here. Methods One and Two both 
give 93.7% compression for a high-resolution (640 by 350) EGA bitmap. Method 
Three gives 99.9% compression for the same resolution. An explanation of these 
compression calculations follows. 

An EGA bitmap occupies 224,000 bits, or 28,000 bytes, of memory. 
Method One carries the run value in bit one and the run length in the remainder of the 
byte, for a maximum run length of 127 bits encoded in each byte. Thus 1764 bytes 
are required to encode a bitmap having only one value, i.e., one mn. Method Two 
carries a maximum run value of 255 bits encoded in each byte; but each byte of 
encoded data must be separated by a "null" byte encoding a run of zero for the 
opposite run value. It takes 879 bytes, doubled, or 1758 bytes to encode the bitmap. 
Method Three, on the other hand, uses a base- 128 mode of encoding long runs. Two 
bytes encodes a maximum run of 16,511 bits; 28 bytes encodes the entire bitmap. 
This yields a compression factor of 1:1000! 

2. Best case -- Color 

For an explanation of color or grey-scale compression in the best case, 
assume the entire file is encoded. Thus only two values are needed; one byte to hold 
the run length (maximum value is 255 pixels) and one or more bytes to hold the run 
value, depending on the number of bits required to encode one pixel. 
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The length of the nin value may actually be disregarded in calculating 
compression. Since it is true that every run of 255 pixels may be encoded by one 
pixel value plus a one-byte run length, then compression is approximately 2:255. But 
to be more specific, if a pixel is contained in eight bits, then every 255 bytes of data 
is encoded with two bytes of data yielding a compression of 99.3%. A 32-bit pixel 
value means that 255 pixels, or 1020 bytes of data, are encoded with five bytes, for 
a compression of 99.5%. 

B. STATISTICAL / HUFFMAN CODES 

In evaluating the effectiveness of statistical encoding, it is important to remember 
the premise on which use of these codes is based. Remember that statistical codes 
are variable length, whereas run-length codes are fixed length. The main idea inherent 
in statistical coding is that symbols which occur more frequently than others in a 
message will be replaced by shorter codes, while symbols which are used less 
frequently will be replaced by longer codes. The concept of entropy, a measure of 
"surprise" at the occurrence of a symbol in a message, is used as the best value, for 
the average bit count in a specific encoding. 

We examine the best, worst and average case for statistical coding in general. 
The conclusions also apply to Huffman codes. In this evaluation, we do not 
distinguish among binary, grey-scale, or color bitmaps. A binary bitmap must be 
considered in blocks of pixels, or patterns, because it makes no sense to encode an 
alphabet containing only two symbols by the statistical method. To do so would create 
a mapping of the source symbols "1" and "0" onto the code symbols "1" and "0" 
respectively. 

1. Best Case -- Statistical Codes 

The formula for calculating entropy is -X PilogjPj, where Pj is the probability 
that the i* symbol in a message will occur. The lower the entropy, the smaller the 
average number of bits required to derive an encoding. The best situation, therefore, 
is the data file which is composed entirely of one character, or one bit pattern. There 
is no surprise as to which symbol will be received next in a transmission. The 
probability of receiving the particular symbol used is one and the entropy is zero! This 
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means that each occurrence of the symbol in the message can be encoded with one bit. 
If the symbol is an eight-bit character or pixel, then compression is 1 - 1/8 = 87.5%. 
If the symbol is a 32-bit pixel, then compression is 96.9%. Both of these upper 
compression limits hold, regardless of the size of the bitmap. 

2. Worst Case - Statistical Codes 

Since the philosophy of statistical encoding is to reduce the average number 
of bits per character by using short codes for highly probable pixel representations, we 
may presume that the advantage will be lost if every character, or pixel value, occurs 
with equal probability. We have seen that one pixel value produces maximum 
compression, and no surprise. Let us look at more than one symbol, transmitted 
randomly, and observe the entropy, or surprise factor, for these situations. Figure 6.1 
does this. It can be seen from the examples in the figure, that if the selected number 
of source symbols in a file is 2”, then the entropy, or best expectation for the average 
number of bits per pixel, is n. It can also be seen that as the number of symbols 
increases, so does the average size of an encoded pixel. Assuming an eight-bit pixel 
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representation, it is clear that for more than 1 28 source symbols with equal distribution, 
there is no benefit to using statistical encoding. This represents the worst case. 

3. Average Case -- Statistical Codes 

It is difficult to derive a generalized "average" compression value for 
statistical encoding. In Chapter VII, where we look at one method of using Huffman 
codes, about 40 actual binary bitmaps were encoded and an average compression value 
was computed for that particular sampling. Because the use of graphics images is so 
varied, ranging from binary satellite map data to multicolor molecular diagrams, 
sampling each particular situation seems a reasonable method of deriving an average 
compression value. 

Another consideration in evaluating statistical encoding is the overhead 
incurred by the encoding method. In Huffman encoding, for example, if the static 
method is used, the program must make two passes of the data to first calculate the 
frequency distribution of the symbols, and then to encode the file. Time to transmit 
the lookup table must be considered, in addition to the I/O time to read the file twice 
and the computing time for encoding. Use of a dynamic Huffman method eliminates 
one pass of the data file and the transmission of the lookup table, but increases the 
compute time required by both host and target machine in order to remain 
synchronized. 

C. RELATIVE ENCODING 

Relative encoding relies on the supposition that bits in any given scan line of a 
bitmap differ little from the previous line. This assumption implies the existence of 
vertical patterns. Once the first line of the bitmap is transmitted, only the relative 
differences of subsequent lines are transmitted. 

With this in mind it is easy to see that the maximum compression is obtained 
when every line of the bitmap is identical. Minimum compression occurs for a 
bitmap where every bit in a scan line is the opposite of the bit immediately above it 
in the previous line. An average compression would fall between these two extremes. 

Although relative encoding is used in facsimile transmissions and may lend itself 
well to compression of a binary image bitmap, it may not prove a satisfactory method 
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for compressing grey scale or color bitmaps due to the greater possibility of vertical 
variation in the map. The advantageous situation occurs in a graph that has large 
filled areas of one shade, or any combination of pixels which provides very little 
vertical change. 

The important issue in any encoding scheme is that the host machine and the 
target machine maintain synchronization, so that at each transmission, the target 
machine knows exactly how to interpret and decode the encoded message being sent. 
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